Quality Assessment and Modulating Factors on Self-Regulatory Behavior in Peer-Assisted Medical Learning

Objectives: Standardized extracurricular skills labs courses have been developed in recent decades and are important approaches in peer-assisted medical learning (PAL). To provide high quality training and achieve effective learning strategies, continuous evaluations and quality assessments are essential. This research aims to evaluate quality data from medical students participating in extracurricular skills labs courses at Ruhr-University Bochum to prospectively optimize concepts and didactical training and standardize processes. Additionally, we set out to assess and quantify drivers that are influencing factors of the self-reflection of competencies. Methods: The analysis was based on a routine assessment of n = 503 attendees of the PAL courses in the skills labs in three consecutive semesters, who voluntarily participated in the evaluation. We analyzed the effects of age, semester and their interaction on the self-reflection of competencies in technical skills courses using moderated regression and simple slope analyses, as previously published. A univariate analysis of variance (ANOVA) with post hoc Tukey HSD testing was used to analyze group means in estimated competencies using IBM SPSS Statistics V.28. Results: An analysis of variance revealed a significant increase in self-assessed competencies when comparing pre- vs. post-course evaluation data in all 35 depicted items (all p < 0.001). A total of 65.5% of the items were adjusted significantly differently, revealing modified self-reflected pre-course levels compared to those stated before. A moderated regression analysis revealed that age (R2 = 0.001, F(1;2347) = 1.88, p < 0.665), semester of study (∆R2 = 0.001, ∆F (1;2346) = 0.012, p < 0.912) and their interaction (∆R2 = 0.001, ∆F (1;2345) = 10.72, p < 0.227) did not explain a significant amount of the variance in self-reflection variance. A simple slope analysis of earlier (b = 0.07, t = 0.29, p < 0.773) and later semesters of study (b = 0.06, t = 0.07, p < 0.941) did not differentiate from zero. Conclusions: The presented evaluation paradigm proved to be a useful tool to encourage students to initiate self-regulatory and self-reflective behavior. The cohesive evaluation of the large cohort of attendees in extracurricular, facultative skills labs courses was helpful in terms of quality assessments and future adaptations. Further evaluation paradigms should be implemented to assess other influencing factors, such as gender, on self-reflection, since age and semester did not explain significant differences in the model.


Peer-Assisted Learning in the Context of Medical Education
Peer-assisted learning (PAL) is defined as a teaching method whereby students ("peers") replace the professional educator, helping teach other students specific skills and to assist them by achieving competencies [1,2]. Especially in medical education, PAL is a frequently used method in curricular as well as extracurricular training at medical faculties [3]. Hereby, the effectiveness of the method was evaluated and PAL was depicted as an important and sufficient approach in medical education [1,4,5]. Although many research approaches have been suggested, earlier investigations postulated to extend knowledge on influencing factors of PAL and evaluate concepts with regard to the different participating actors and perspectives [3].

The National Competence-Based Catalogue of Learning Objectives for Medicine (NKLM 2.0)
The guidelines of the National Competence-based Catalogue of Learning Objectives for Medicine (NKLM 2.0) of the German Medical Faculty Association (MFT) highlight competence-based practical medical education (CBME) as the main and most important approach [6][7][8]. To implement competency-based educational concepts based on recommendations in the NKLM, didactic strategies have evolved aiming to generate improved practical teaching competencies [9]. Hereby, investigations have focused on the identification of practical learning goals to standardize the teaching methods of practical approaches in medical faculties [10]. In this context and for the training of clinical competencies in a riskless situation prior to the application in patients, skills laboratories (skills labs) have been developed as equipped practical rooms providing medical students and teachers adequate learning situations and a beneficial atmosphere to improve their competencies [11].

Training in Skills Labs
In Germany, most medical faculties (>90%) have established skills labs in recent decades. However, a need for improving methods and studies in terms of standardization, quality assessments and developing teaching concepts has arisen regarding PAL concepts in the skills labs landscape [12]. These needs have not only been raised by medical faculties and educators but also in nursing and other health professional educational contexts [13,14].
Although many concepts have been developed in the past in terms of evaluated self-regulated and competency-based learning among participating students [15][16][17][18][19][20] and research has focused on the development and implementation of PAL approaches as a common didactical method, an urgent need to standardize concepts, didactical methods and professional teachers as well as content training has been described [12,[21][22][23].
A focus has been placed on the assessment of evaluation concepts and data coming from participants in practical, peer-assisted, extracurricular training in skills courses [2,24]. To evaluate reflected competencies, recently, approaches have focused not only on the selfassessment of competencies before and after course participation, but additionally on the imaginary (self-reflected) view prior to course participation [20,25]. Hereby, an appropriate self-reflection of competences-as a non-technical and metacognitive skill-is not only important in terms of patient safety and the learning process, but additionally because it is named as part of the NKLM, so that, as other researchers postulated, medical students can be longitudinally trained [26,27].
We set out to evaluate quality assessment data from medical students participating in extracurricular skills labs courses at the Ruhr-University Bochum. With this, we aimed to identify certain components which might prospectively optimize concepts and didactical training as well as standardize processes. This might be transferable to other medical faculties. This research quantifies influencing factors of self-assessed properties with regard to self-regulatory behavior of practical competencies. We evaluated technical concepts to identify course-specific aspects, since earlier interventions analyzed the self-regulatory behavior of participants in the first aid and physical examination course [20,28].

Evaluation Data from Peer-Assisted Skills Labs Courses
We investigated data from n = 503 attendees of PAL courses at the Ruhr-University Bochum. We analyzed data from three semesters between 2018 and 2020 from students voluntarily participating in the evaluation of our skills labs. Participants were from all different semesters of the entire medical program of the faculty. As a first approach, we analyzed the demographic data of participants.
Standardized peer-assisted courses were exclusively delivered by peers after they had been trained by didactical and medical experts in the field of the specific course content. Therefore, structured course manuals were implemented.

The Evaluation Paradigm
The evaluation concept consisted of a self-assessment of competencies before (1: pre), after (2: post), and after the course with a retrospective assessment of competencies before participating in the course (3: post-pre) as previously published [20,25]. Next to group comparisons of prevs. post-, prevs. post-pre and postvs. post-pre-evaluation data, we analyzed the influence of age, semester of study and its interaction on the self-reflection of competencies, depicted as the concordant self-reflection as earlier described [20]. The latter was defined as the difference of the post-preminus preassessment. Negative scores indicate that participants reveled a more negative (critical) assessment of their own competencies before the course and afterwards retrospectively adjusted these scores to better self-assessed competencies [20]. Higher positive data revealed that participants evaluated themselves as having better self-assessed competencies before the course than retrospectively. Here, we assessed evaluations from technical skills courses (n = 335 participants), implementing concepts for teaching blood collection, surgery, peripheral indwelling catheters, changing bandages and central indwelling catheters.
Evaluation included, in each course, a survey of different course-specific items and a self-assessment, documented using a six-point Likert scale differentiating between strongly agree = 1 and strongly disagree = 6 as the most severe expressions, as previously published [20,28]. Students were asked if they felt confident in dealing with the respective specific competence; lower numbers (1) meant better competencies and higher numbers (6) revealed lower competencies. The overall evaluation was also assessed by a six-point Likert scale; however, it was oriented with a score of 1 being very good and 6 being very poor. Data were evaluated as part of the routine course assessments.
Pseudonymized evaluation data were assessed routinely as part of the quality and evaluation assessments and collection and analysis were conducted anonymously.

Statistical Analysis
We implemented a statistical approach, as previously published and described in our earlier work. Group means and standard deviation were evaluated using a univariate analysis of variance (ANOVA) with post hoc Tukey HSD testing for pairwise analysis of different estimates on the self-assessment of competencies in IBM SPSS Statistics V.28. Adjustment for multiple testing was applied using Bonferroni corrections. Dependent variables were tested for normal distribution using the Kolmogorov-Smirnov test. The homogeneity of variances was asserted using Levene's Test. For unequal variances, values were reported with Welch's test. Chi-square tests were used for analyses of categorical and descriptive variables. A moderated regression analysis was conducted to analyze the interaction effect of age and semester of study, as well as their interaction, on the concordant self-reflection of competencies in technical course concepts. Simple slope analyses were calculated to illustrate the effects [20].

Demographic Data of Participants in Peer-Assisted Extracurricular Course Concepts
The following table provides participant ages, semesters of study and whether participants were professionally trained previously or had participated in the courses before (Table 1). Regarding different course domains, most participants attended one of the technical courses (66.4% of all participants). In total, n = 503 students participated in courses and the evaluation ( Table 2). An analysis of participants with regard to the study semester revealed that most participants attended during their fifth semester (24.3% of all participants) and the majority of all participants were 20 (19.5% of all participants) or 21 (18.5% of all participants) years old ( Figure 1). Regarding different course domains, most participants attended one of the technical courses (66.4% of all participants). In total, n = 503 students participated in courses and the evaluation ( Table 2). An analysis of participants with regard to the study semester revealed that most participants attended during their fifth semester (24.3% of all participants) and the majority of all participants were 20 (19.5% of all participants) or 21 (18.5% of all participants) years old ( Figure 1).

Evaluation Regarding Quality Assessment Data
As a first approach, we assessed data from the general and non-course-specific part of the presented evaluation paradigm, documenting attendees satisfaction and overall impression with regard to their course participation. Data were surveyed in ten items, as well as two concluding questions, with regard to the overall course evaluation (the results are documented in Table 3).

Analysis of Self-Reflected Competencies in Technical Course Concepts
Following this, we assessed evaluation data from N = 335 participants in the technical course concepts with regard to attendees pre-, post-and post-pre-evaluation. In all 35 out of 35 assessed domains from the five course concepts, participants revealed a significant increase in self-assessed competencies with regard to pre-vs. post-evaluation data (all p < 0.001). The same differences were depicted for the post-pre-vs. post-data (all p < 0.005).

Evaluation Regarding Quality Assessment Data
As a first approach, we assessed data from the general and non-course-specific part of the presented evaluation paradigm, documenting attendees' satisfaction and overall impression with regard to their course participation. Data were surveyed in ten items, as well as two concluding questions, with regard to the overall course evaluation (the results are documented in Table 3).

Analysis of Self-Reflected Competencies in Technical Course Concepts
Following this, we assessed evaluation data from N = 335 participants in the technical course concepts with regard to attendees pre-, postand post-pre-evaluation. In all 35 out of 35 assessed domains from the five course concepts, participants revealed a significant increase in self-assessed competencies with regard to prevs. post-evaluation data (all p < 0.001). The same differences were depicted for the post-prevs. post-data (all p < 0.005).
Regarding the prevs. post-pre-evaluation, a significant change in the self-assessment was observed in 23 out of 35 domains (Table 4). Table 4. Analysis of self-reflection of competencies with regard to course-specific domains (Pre-, Post-, and Post-Pre-Scores: 1 = very good to 6 = very poor).

Analysis of Significantly Different Pre-vs. Post-Pre-Comparisons
The comparison of prevs. post-pre-data was assessed as self-regulatory behavior. Out of 35 domains, students significantly evaluated post-pre-levels differently in 23 domains, revealing a retrospectively adjusted pre-competency level. Remarkably, in 22 of these items, attendees retrospectively assessed their pre-competencies as better than before. In the following, all significantly differentiating prevs. post-pre-items are depicted. The differences in post-pre minus pre-data were analyzed, revealing negative scores as a more critical assessment before the course, which were afterwards corrected to improved prelevels ( Figure 2).
Healthcare 2023, 11, x FOR PEER REVIEW 7 of 12 critical assessment before the course, which were afterwards corrected to improved prelevels ( Figure 2).

Moderating Effects of Age, Semester of Study and Their Interaction on the Self-Regulatory Behavior of Competencies
As a further approach, we set out to analyze the effects of age, study semester and their interaction on self-regulatory behavior using a moderated regression analysis. To analyze the self-regulatory behavior depicted as the difference in post-pre-minus pre-assessment, we assessed data from all five assessed technical courses, revealing an overall number of 2349 reflected items. The moderated regression analysis revealed that age (R 2 = 0.001, F(1;2347) = 1.88, p < 0.665), semester of study (∆R 2 = 0.001, ∆F (1;2346) = 0.012, p < 0.912) and the interaction of both (∆R 2 = 0.001, ∆F (1;2345) = 10.72, p < 0.227) did not explain a significant amount of variance in the self-reflected variance (F = 0.55, p < 0.646). A simple slope analysis was conducted to illustrate the effects in a figure. Slopes for a tendentially low semester of study (b = 0.07, t = 0.29, p < 0.773) and high semester (b = 0.06, t = 0.07, p < 0.941) did not significantly differentiate from zero. Illustrations revealed that participants tended to have a more critical assessment of their competencies before the course and this was afterwards retrospectively adjusted to better self-assessed pre-competencies. This was, however, independent from the age and semester of participants (Figure 3).

Moderating Effects of Age, Semester of Study and Their Interaction on the Self-Regulatory Behavior of Competencies
As a further approach, we set out to analyze the effects of age, study semester and their interaction on self-regulatory behavior using a moderated regression analysis. To analyze the self-regulatory behavior depicted as the difference in post-preminus pre-assessment, we assessed data from all five assessed technical courses, revealing an overall number of 2349 reflected items. The moderated regression analysis revealed that age (R 2 = 0.001, F(1;2347) = 1.88, p < 0.665), semester of study (∆R 2 = 0.001, ∆F (1;2346) = 0.012, p < 0.912) and the interaction of both (∆R 2 = 0.001, ∆F (1;2345) = 10.72, p < 0.227) did not explain a significant amount of variance in the self-reflected variance (F = 0.55, p < 0.646). A simple slope analysis was conducted to illustrate the effects in a figure. Slopes for a tendentially low semester of study (b = 0.07, t = 0.29, p < 0.773) and high semester (b = 0.06, t = 0.07, p < 0.941) did not significantly differentiate from zero. Illustrations revealed that participants tended to have a more critical assessment of their competencies before the course and this was afterwards retrospectively adjusted to better self-assessed pre-competencies. This was, however, independent from the age and semester of participants (Figure 3). Figure 3. Analysis of interaction effects of age and semester of study on the self-regulatory assessments in technical courses illustrated as simple slopes (0 = no differences between pre-and post-preevaluations; negative scores = a more critically pre-assessment, retrospectively adjusted to better precompetencies).

Discussion
Competency-based and practical medical learning are important approaches in current medical education paradigms that require standardized concepts and a longitudinal evaluation in terms of continuous quality assessments and objectifiable principles [29,30]. The presented investigation aimed to evaluate quality data from course attendees from a large student cohort participating in extracurricular PAL at the Ruhr-University Bochum to objectify the current satisfaction of participants. Furthermore, we aimed to objectify the influencing factors of self-regulatory learning and depict course-specific factors. Hereby, the self-regulatory behavior and self-reflection of skills is known as a metacognitive process and is an essential tool in adequate learning processes and successful medical learning [27]. A better understanding of influencing factors on self-assessment and self-reflection was previously determined as fundamental for effective practical skill training; however, research has determined multiple influencing and partly unknown factors that may play a bias or modulating role [31].
As a first result, we identified by an analysis of demographic attendees data that participants of the extracurricular courses predominantly attended in their first five semesters of study (about 74% of all participants), whereby more than one-third (38%) was 20 or 21 years old. With regard to earlier investigations, which described that age is associated with a better developed personality and trained emotional regulative processes [32], this could conversely mean that students at a comparatively younger age are still in the developmental and learning process, meaning practical teaching methods may positively influence older health professionals. Thereafter, the course takes place at an appropriate time. At the same time, the findings suggest that further concepts for students in later semesters, implementing, e.g., more complex skills, should be offered to expand the training courses in later semesters or to older students. The course concept regarding the central venous catheter insertion-as published earlier-is a more complex course which might be a suitable approach and should further be expanded. [16,33] A further assessment of the overall quality evaluation data revealed that participants were satisfied after participating in the courses (99.8% recommended the course), which is explainable by the standardized concepts and the earlier continuous evaluation strategy. Standardized  Figure 3. Analysis of interaction effects of age and semester of study on the self-regulatory assessments in technical courses illustrated as simple slopes (0 = no differences between preand post-pre-evaluations; negative scores = a more critically pre-assessment, retrospectively adjusted to better pre-competencies).

Discussion
Competency-based and practical medical learning are important approaches in current medical education paradigms that require standardized concepts and a longitudinal evaluation in terms of continuous quality assessments and objectifiable principles [29,30]. The presented investigation aimed to evaluate quality data from course attendees from a large student cohort participating in extracurricular PAL at the Ruhr-University Bochum to objectify the current satisfaction of participants. Furthermore, we aimed to objectify the influencing factors of self-regulatory learning and depict course-specific factors. Hereby, the self-regulatory behavior and self-reflection of skills is known as a metacognitive process and is an essential tool in adequate learning processes and successful medical learning [27]. A better understanding of influencing factors on self-assessment and self-reflection was previously determined as fundamental for effective practical skill training; however, research has determined multiple influencing and partly unknown factors that may play a bias or modulating role [31].
As a first result, we identified by an analysis of demographic attendees' data that participants of the extracurricular courses predominantly attended in their first five semesters of study (about 74% of all participants), whereby more than one-third (38%) was 20 or 21 years old. With regard to earlier investigations, which described that age is associated with a better developed personality and trained emotional regulative processes [32], this could conversely mean that students at a comparatively younger age are still in the developmental and learning process, meaning practical teaching methods may positively influence older health professionals. Thereafter, the course takes place at an appropriate time. At the same time, the findings suggest that further concepts for students in later semesters, implementing, e.g., more complex skills, should be offered to expand the training courses in later semesters or to older students. The course concept regarding the central venous catheter insertion-as published earlier-is a more complex course which might be a suitable approach and should further be expanded. [16,33] A further assessment of the overall quality evaluation data revealed that participants were satisfied after participating in the courses (99.8% recommended the course), which is explainable by the standardized concepts and the earlier continuous evaluation strategy. Standardized concepts can be used as a useful starting point to integrate innovative learning methods or technical adaptations in future.
The main focus was set on the evaluation of self-assessed competencies, whereby the evaluation concept proved that participants subjectively benefitted from course participation. This was documented in all assessed 35 evaluation items.

Limitations
However, as a limitation of the presented data here, no objectifiable measures of competencies are given; thus, future approaches should expand the focus to practical competency increases, e.g., by assessing the skills of participants and discrepancies between the self-perceived competencies. An appropriate approach could be to evaluate competencies in summative tests like "Objective Structured Clinical Examinations (OSCE)" which is currently being considered for implementation in curricular medical trainings with regard to the implementation of new licensing regulations for medical professionals. This limitation mainly concerns the assessment of trained competencies, whereby an assessment of self-regulatory behavior itself seems difficult to evaluate.
As a main advantage of the presented data, not only were prevs. post-assessments given, but also self-reflected pre-competencies. Hereby, about 65.5% of the assessments significantly differed when compared to the pre-data. This is nearly congruent with previous assessments of first aid and examination courses, whereby 72.7% differed significantly. [20] The fact that more than two-thirds of the participants retrospectively adjusted their precompetencies suggests that the evaluation paradigm forced students to make use of selfreflective or self-regulated behavior. Since the ability for self-reflection was previously postulated as an important value for older health professionals, the evaluation concept should additionally be expanded in the courses as a useful tool to train medical students in self-regulatory competencies [34].
Remarkably, in nearly all assessments (22 out of 23 items), students corrected their initial pre-levels retrospectively to better pre-course levels (depicted in the post-pre evaluation). This might be explainable at least partly by gender. An analysis of demographic data revealed that more than two-thirds (67.9%) of the attendees were female. Earlier research postulated that gender has an influence on self-regulatory behavior. [35][36][37] Hereafter, it might be conceivable that the majority of female students initially had a more critical self-assessment of competencies and noticed after participating in the course that the earlier assessed competency was underestimated with regard to the specific item. Future research should therefore focus additionally on gender-specific aspects to extend the knowledge of influencing factors. Notably, no significant effects were observed based on the influence of age, semester of study and their interaction on the reflected self-assessment, which is contrary to earlier investigations of other concepts [20]. It is reasonable that course-specific, gender-specific and other undetected aspects may have had an influence, and overlapping effects of age, semester and their interaction were amended. In particular, the technical course format consists of shorter units, whereby examinations and first aid courses are held in different timed modules on different days. This might have had an influence as well.
As a further limitation, other-and to date unknown-factors may be involved which have not been further assessed, since self-reflection was evaluated as part of the established routine evaluation design and no further specific questionnaires were implemented. Specific scales have previously been described for the assessment of self-reflection competencies, such as the "Self-Regulation of Learning Self-Report Scale (SRL-SRS)", and should therefore be implemented in future assessments. [38] The depicted illustrated graphs in the simple slope analysis proved to be a useful psychometric tool to document that participants tended to have a more critical assessment of their own competencies before course participation, which afterwards adjusted to better pre-course competencies, regardless of age and semester of study, which is therewith partly contrary to earlier findings [20].

Conclusions
The presented evaluation paradigm is a useful tool to encourage students to initiate self-regulatory and self-reflective behavior. The first cohesive evaluation of the presented large cohort of attendees in our extracurricular, facultative skills labs was helpful in terms of a quality assessment and future adaptations. Further longitudinal evaluation paradigms will be implemented with standardized questionnaires for an assessment of self-regulatory skills and should objectify trained practical skills. Other influencing factors of self-reflection of skills in medical education, such as gender, should be analyzed, since age and semester did not explain significant differences in the analyzed model.  Institutional Review Board Statement: Not applicable since evaluation data were assessed as part of routine course assessments; analysis and collection were conducted retrospectively and anonymously. Data were assessed for the internal quality assessment of the extracurricular course concepts, which did not meet the definition of research for which review or approval was required. The course attendees voluntarily took part in the routine evaluation concept, and data were pseudonymized.
Informed Consent Statement: Informed consent was obtained from all subjects involved in the study.

Data Availability Statement:
The data that support the findings of this study are available from the corresponding author upon reasonable request. The data are not publicly available.